test_bot/formulae: only run subset of brew tests for portable Ruby#23113
Conversation
Currently tests fail on all runners (Tier 1 Linux and Tier 3 macOS). It is difficult to align the setups used between Homebrew/brew and Homebrew/core so can limit to basic sanity of the cask and formula tests.
There was a problem hiding this comment.
Pull request overview
This PR adjusts test_bot’s portable Ruby workflow to avoid failures across unsupported/less-aligned runner environments by (a) restricting when brew style runs and (b) reducing the scope of brew tests executed for portable Ruby.
Changes:
- Run
brew styleonly when Homebrew considers the current environment a “Tier 1 configuration”. - Restrict
brew teststo a subset via--only=cask,formulafor portable Ruby runs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| test "brew", "install-bundler-gems", "--groups=all" | ||
| test "brew", "vendor-gems", "--non-bundler-gems", "--no-commit" | ||
| test "brew", "tests", "--online", "--coverage" | ||
| test "brew", "tests", "--online", "--coverage", "--only=cask,formula" |
There was a problem hiding this comment.
Why do we have tests failing on Tier 1 Linux runners? Any ideas?
There was a problem hiding this comment.
Currently failing on Gzip unpack test. I've tried a few changes to debug and see if chown could help:
- test/unpack_strategy: run extract with verbose #23070
- test/unpack_strategy: use Mktemp to set group #23075
Still failing on the sticky bit. I don't think we ever add it ourselves, so I'd expect the default /var/tmp sticky is causing issues in our nested directory.
Since I haven't been able to reproduce yet, I'm not able to figure out the exact conditions this happens. No issues running brew tests --only unpack_strategy/gzip on Linux VM where I spun up a Docker container (as close as I can get to CI).
There was a problem hiding this comment.
Ok, thanks. Can we just skip that specific test in brew tests itself so we can run the test?
There was a problem hiding this comment.
Can explore again since we no longer are blocking any PRs.
I think root case is actually from running brew vendor-gems prior to tests:
brew/Library/Homebrew/dev-cmd/vendor-gems.rb
Lines 48 to 50 in df0164d
There was a problem hiding this comment.
|
Thanks for unblocking us here @cho-m ❤️ |
Currently tests fail on all runners (Tier 1 Linux and Tier 3 macOS). It is difficult to align the setups used between Homebrew/brew and Homebrew/core so can limit to basic sanity of the cask and formula tests.
Also update
brew styleto run on Tier 1-only so that we don't have to manually update formula list and can keep independent frombrew styleimplementation details.Portable Ruby PRs have been broken for over a month now. Given we now have a new Ruby update (along with OpenSSL, etc.), this should unblock us:
brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?